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ABSTRACT 



Background: Cochrane reviews are regarded as scientifically rigorous, yet a review's time to publication can be 
affected by factors such as the statistical significance of the findings. When this happens, misrepresentation of the 
literature and subsequent inappropriate decisions may result. We aimed to examine the factors associated with the 
time to publication of Cochrane reviews. 

Methods: Review protocols published in issue 2, 2000, of the Cochrane Database of Systematic Reviews were includ- 
ed in this analysis if the corresponding review was published by issue 1, 2008. We used univariable and multivariate 
analyses to examine the time from publication of the protocol to publication of the first review and review-related 
factors predicting the time to publication. 

Results: Of 118 eligible protocols published in issue 2, 2000, we identified 93 Cochrane reviews that had been pub- 
lished by January 2008. Of these, 36 (39%) were updates. The median time to publication was 1.63 years (range 
0.15-7.31 years). A change in authors between publication of the protocol and publication of the final review was 
associated with longer time to publication (p = 0.002), whereas updated reviews were associated with shorter time to 
publication (p = 0.030). 

Interpretation: In our study, 79% of the Cochrane protocols were published as a final review, and some Cochrane 
reviews took over 7 years from publication of the protocol to publication of the final review. Strategies to increase the 
number of published Cochrane reviews and decrease the time to publication should be considered, such as providing 
support to reviewers when a change in authorship occurs. 
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PUBLICATION BIAS OCCURS WHEN STUDIES WITH Cer- 
tain characteristics (e.g., statistically significant 
results) have a greater likelihood of being pub- 
lished, 1 and of being published more quickly, 2 than stud- 
ies without those characteristics. We previously examined 
publication bias through a retrospective cohort study in 
which we followed Cochrane reviews for approximately 
9 years and found that the majority of studies were pub- 
lished (301/372 [8i%]). 3 In that study, we abstracted data 
from Cochrane protocols and used the information to 
determine which factors predicted publication of the cor- 
responding Cochrane reviews. We found that the time to 
publication was shorter for reviews that were subsequent- 
ly updated (hazard ratio for no update v. update 1.80, 95% 
confidence interval [CI] 1.39-2.33) and longer for reviews 
with 2 published protocols, indicating changes to the re- 
view plan (hazard ratio for 1 protocol v. 2 protocols 0.33, 
95% CI 0.12-0.90). These data were based on 
comparing published Cochrane reviews and un- 
published Cochrane reviews by examining differ- 
ences observed in their protocols. 

For the study reported here, we aimed to 
examine review-related factors (e.g., statistic- 
ally significant results) associated with the time 
to publication of Cochrane reviews. Since such 
review factors are not easily obtained from un- 
published Cochrane reviews (i.e., this informa- 
tion could be acquired only by contacting the 
Cochrane review authors), we included only 
published Cochrane reviews in this study. 

Methods 

To allow a reasonable time for publication of the 
review, we first identified all new protocols that 
were published in issue 2, 2000, of the Cochrane 
Database of Systematic Reviews. We tracked the 
status of these protocols by searching the Coch- 
rane Database of Systematic Reviews until Janu- 
ary 23, 2008 (issue 1, 2008) and by contacting 
the corresponding authors or the Cochrane 
Review Group coordinator. We excluded Coch- 
rane protocols that were split into more than 1 
Cochrane review, taken over by another review 
group, published in the same issue as the final 
Cochrane review, published after publication of 
the corresponding review or published for the 
second or subsequent time in issue 2, 2000. 3 

We developed and pilot-tested a comprehen- 
sive data abstraction form. One of us (ACT) used 
this form to abstract data from the published 



Cochrane reviews, such as characteristics of the pub- 
lished review (e.g., number of authors), methodology 
used (e.g., number of primary outcomes, inclusion of un- 
published material, assessment of publication bias) and 
other factors (e.g., funding, number of updates). 

In addition, we classified the results and conclusions 
using a system reported elsewhere. 4 Briefly, we classified 
the results of each review as negative, not statistically sig- 
nificant (unfavourable toward treatment intervention, p > 
0.05); negative, statistically significant (unfavourable to- 
ward treatment intervention^ < 0.05); neutral (effect size 
between 0.95 and 1.05 with the CI crossing 1); positive, 
not statistically significant (favourable toward treatment 
intervention, p > 0.05); positive, statistically significant 
(favourable toward treatment intervention, p < 0.05); 
and indeterminate (unable to judge because, for example, 
the Cochrane review listed 10 primary outcomes, which 

Cochrane protocols from 
issue 2, 2000 
n = 130 



Protocols excluded n = 12 

• Duplicate n = 2 

• Final review published before protocol n = 2 

• Protocol and review published in same issue n = 2 

• Protocol published before issue 2, 2000 n = 3 

• Another group took over review n = 3 



Relevant Cochrane protocols 
n= 118 



Protocols not published as Cochrane review n = 25 

• Protocol withdrawn because out of date n = 3 

• Protocol still active n = 13 

• Protocol or review not found in Cochrane Library, and 
corresponding author did not respond to inquiry n = 6 

• Cochrane Collaboration rejected review n = 1 

• Corresponding author reported that review team lacked 
time or interest to complete review n = 1 

• Corresponding author reported that review not 
completed because lead author switched jobs n = 1 



Cochrane protocols published 
as Cochrane reviews 
n = 93 

(includes 1 withdrawn review) 



Figure 1 : Study selection 
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Table 1: Cochrane review characteristics (n = 93*) 



Characteristic No. (%)* 
Descriptive characteristics 

Population examined 

Neonates only 7 (") 

Children only ^ ^' 

Adults only 20 < 22 ' 
Women only 

Men only 1 0) 

Children and adolescents ^ v) 

Adolescents and adults ^ v'') 

Adolescents, adults and elderly people ^ C ' 

Adults and elderly people ^ C ' 

All ages 47 < 51 > 

Number of authors, median (range) 3 (1-8) 

Change in authors between protocol and corresponding review 49 (53) 

Review had 2 protocols 2 (2) 

Review had 2 unique Cochrane identification numbers 9 (10) 

Number of studies included, median (range) 5 (0-84) 
Number of participants included, median (range) 410 (0-109 394) 

Number of pages in pdf review file, median (range) 23 (5-183) 

Methodological characteristics 

Type of reports included in the review 

Observational only " (") 

Experimental and quasi-experimental only "9 (96) 

Both observational and experimental 4 

Number of databases searched, median (range) 4 (1-17) 

A primary outcome was reported 75 (81 ) 
Languages included 

English only 0 (0) 

Mixed languages only 2 (2) 

All languages 49 (53) 

Not reported 42 (45) 
Status of study reports included 

Published material only 2 (2) 

Published and unpublished material 69 (74) 

Not reported 22 (24) 

Meta-analysis performed for primary outcome 54 (58) 

Publication bias was assessedt 9 (17) 

Publication bias affected resultst 1 (2) 

Heterogeneity was assessedt 54 (100) 

Heterogeneity affected resultst 8 (15) 

Classification of results and conclusions 

Resultst 

Indeterminate ' ( 17 ' 

Negative, not statistically significant '® ('") 

Negative, statistically significant " ^' 

Neutral 7 < 13 ' 

Positive, not statistically significant " ' 2 ^ 

Positive, statistically significant ^ 7 (31) 
Conclusions 

Indeterminate 40 (43) 

Negative 10(11) 

Neutral 24 (26) 

Positive 19 (20) 

Other factors 

Corresponding author was a health care provider 2 ^ ' 27 ' 

Number of reviews that were updated 36 (39) 

Number of reviews with funding 74 (80) 
* Except where otherwise noted. 



t The denominator is the number of reviews for which a meta-analysis was conducted (n = 54). 



had different results). The conclusions were 
classified as positive (authors stated that 
there is evidence of effectiveness), neutral 
(authors stated that there is no evidence of 
effectiveness or they reported no opinion), 
negative (authors advised against use of the 
intervention or it was not recommended) or 
indeterminate (authors stated that there is 
insufficient evidence or that more research 
is required). 

We first conducted univariable analy- 
sis for the published Cochrane reviews by 
linear regression. The data were skewed 
and did not follow a normal distribution. 
Therefore, we used the log transformation 
of the time between the "most recent sub- 
stantive amendment date" of the protocol 
and the published review to analyze the 
time variable. 5 The characteristics reported 
in Table 1 were analyzed using linear regres- 
sion. They were entered into a univariable 
analysis, and those that were statistically 
significant as well as those that we defined 
a priori as being important (i.e., results 
classified as favourable v. unfavourable and 
all others v. unfavourable; conclusions clas- 
sified as negative v. positive and all others 
v. positive; funding; number of authors) 
were included in the multivariate linear re- 
gression analysis. Since only published re- 
views were included, none of the data were 
censored (i.e., losses to the sample that oc- 
curred prior to the final outcome) or mis- 
sing. Therefore, we did not perform a Cox 
proportional hazards model, which is com- 
monly used to estimate time-related events 
and takes into account censored data. Sta- 
tistical analyses were conducted with SAS, 
version 9.0 (SAS Institute, Cary, NC). 

Results 

A total of 130 new Cochrane protocols ap- 
peared in issue 2, 2000, of the Cochrane 
Database of Systematic Reviews. Twelve 
protocols were excluded because they did 
not meet the inclusion criteria, and 93 
(79%) of the remaining 118 protocols were 
published as Cochrane reviews and were 
included in this study, including 1 with- 
drawn review (Fig. 1). 



Open Medicine 2009;3(4):e210-E214 



Research 



Tricco et al 



A primary outcome was reported in 75 (81%) of the 
93 included reviews. More than half of the reviews in- 
cluded studies published in any language (49/93 [53%]), 
and nearly three-quarters included both published and 
unpublished material (69/93 [74%]) (Table 1). Publica- 
tion bias was found to affect the results in only 1 (11%) 
of the 9 Cochrane reviews that assessed the publication 
bias of included studies. Of the 54 Cochrane reviews in 
which a meta-analysis of the primary outcome was per- 
formed, the largest category for classification of results 
was for studies with statistically significant positive re- 
sults (17/54 [31%]), whereas the largest category for clas- 
sification of conclusion statements was for studies with 
indeterminate conclusions (40/93 [43%]). Only 36 (39%) 
of the 93 Cochrane reviews were subsequently updated, 
and 74 (80%) reported a funding source. 

The median time to publication was 1.63 years (range 
0.15-7.31 years). Four factors were significant predict- 
ors of publication, by univariable analysis: presence of 2 
protocols, subsequent updating of the review, change in 
authorship between publication of the Cochrane protocol 
and publication of the review, and number of included 
studies (all p < 0.05). As noted in the Methods section, 
we included an additional 4 factors in the multivariate 
analyses on the basis of a priori consideration. Of these 
8 factors, a change in authorship between publication 
of the protocol and publication of the review was asso- 
ciated with longer time to publication (p = 0.002), and 
updating of the review was associated with shorter time 
to publication (p = 0.030) (Table 2). 

Interpretation 

Our results indicate that 79% of the Cochrane protocols 
were published as a final review, and that a change in 



Table 2: Factors associated with the time to publication of Cochrane 
reviews, as determined by multivariate analysis 



Factor 


Estimate* 


Standard error 


p value 


Two protocols (no v. yes) 


-0.810 


0.467 


0.086 


Author change (no v. yes) 


-0.426 


0.134 


0.002 t 


Number of authors 


0.034 


0.051 


0.504 


Funding (no v. yes) 


0.280 


0.167 


0.097 


Number of included studies 


0.007 


0.005 


0.145 


Results (favourable v. unfavourable) 


0.018 


0.274 


0.948 


Results (all others v. unfavourable) 


-0.167 


0.244 


0.495 


Conclusions (negative v. positive) 


0.059 


0.276 


0.832 


Conclusions (all others v. positive) 


-0.109 


0.188 


0.562 


Update (no v. yes) 


0.305 


0.138 


0.030 t 



* A negative value indicates longer time to publication, and a positive value indicates 

shorter time to publication. 
tp<0.05. 



authorship between publication of the Cochrane proto- 
col and publication of the final review was associated 
with longer times to publication. Since the Cochrane Col- 
laboration is internationally recognized for how it con- 
ducts systematic reviews, it would be beneficial if all of 
the published protocols were also published as reviews. 
The Cochrane Collaboration should therefore consider 
strategies to improve the publication rate of Cochrane 
reviews and to decrease their time to publication. 

Our results are inconsistent with research examining 
publication bias of individual studies (e.g., trials). 2,6 ~ 10 
In those studies, statistically significant results and 
funding were associated with publication. Given that 
our study did not find such associations, it appears that 
publication patterns differ between Cochrane reviews 
and trials. This conclusion is consistent with other re- 
search on publication bias of systematic reviews. In 
a survey of authors of published systematic reviews, 
participants reported that non-statistically significant 
results were not a major reason why their systematic re- 
views were unpublished; however, 65% (195/300) also 
reported that informative or statistically significant 
results were the "most significant" or a "significant" 
facilitator to publishing their reviews. 11 Furthermore, 
the funding source was not associated with the time to 
publication of Cochrane reviews in our previous retro- 
spective cohort study. 3 However, our analysis of statis- 
tically significant results in the current study was based 
on only a little over half of the sample, since a meta- 
analysis had been conducted for only 54 (58%) of the 93 
Cochrane reviews. 

This study, like our previous one on a similar topic, 3 
had some limitations. A single investigator abstracted all 
of the data, which might have led to inaccuracies. Assess- 
ing the primary outcome, results and 
conclusions of the Cochrane reviews 
was often difficult, and there may have 
been errors in these assessments. 4 

In conclusion, 79% of the Cochrane 
protocols were published as a final re- 
view, and some Cochrane reviews took 
over 7 years from publication of the 
protocol to publication of the final re- 
view. Strategies to increase the number 
of published Cochrane reviews and de- 
crease the time to publication should 
be considered. Such strategies might 
include providing additional support to 
reviewers when a change in authorship 
occurs. 
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